Recently, with the advent of the age of multimedia which handles audio, images and other pixel values in an integrated manner, conventional information media, such as newspapers, journals, televisions, radios, and telephones, through which information is carried to people, have come under the scope of multimedia. Generally, multimedia refers to a representation in which not only text but also graphics, audio, particularly images, and/or others are simultaneously associated with one another. The information for the above conventional information media must first be digitized before it can be handled as multimedia information.
However, the estimated amount of the multimedia information as digital data is only 1 or 2 bytes per character of text, but 64 Kbits per second of audio (telephone quality), and 100 Mbits or higher per second of video (at current television receiver quality). It is therefore not practical to handle these massive amounts of information in digital form on the above information media. For example, video telephony services are available over Integrated Services Digital Network (ISDN) lines with a transmission speed of 64 Kbit/s to 1.5 Mbit/s, but video for a television and a camera cannot be sent as it is over the ISDN lines.
Data compression therefore becomes essential. Video telephony services, for example, are implemented using video compression techniques standardized in International Telecommunication Union, Telecommunication Standardization Sector (ITU-T) Recommendations H.261 and H.263. Using the data compression techniques defined in MPEG-1, image information can be recorded together with audio information on a conventional audio compact disc (CD).
The Moving Picture Experts Group (MPEG) is an international standard for compressing video signals, and has been standardized by the International Organization for Standardization and the International Electrotechnical Commission (ISO/IEC). MPEG-1 is a standard that enables transmission of a video signal at 1.5 Mbps, that is, compression of information in a television signal approximately to a hundredth part of its original size. The moderate image quality is targeted in MPEG-1 because the transmission speed for MPEG-1 moving images is limited to approximately 1.5 Mbit/s. Therefore, MPEG-2, which has been standardized to meet the demand for higher image quality, enables transmission of a video signal at 2 to 15 Mbit/s to satisfy television broadcast quality.
Furthermore, the working group (ISO/IEC JTC1/SC29/WG11) that has worked on the standardization of MPEG-1 and MPEG-2 has standardized MPEG-4 that achieved a compression rate higher than those of MPEG-1 and MPEG-2. MPEG-4 not only enables coding, decoding, and operating on a per object basis, but also introduces a new capability required in the multimedia age. MPEG-4 achieved a compression rate higher than those of MPEG-1 and MPEG-2, and further enables coding, decoding, and operating on a per object basis.
At first, MPEG-4 had been developed for the purpose of the standardization of a coding method for a lower bit rate. Then, it was extended to a more versatile coding method including a method for coding even interlaced images at a high bit rate. The MPEG-4 AVC (ITU-T H.264) has been standardized as a method for coding an image at a higher compression rate through collaboration between the ISO/IEC and the ITU-T.
Here, an image signal can be consecutive pictures (also referred to as frames or fields) that are groups of pixels at a same time. Since pixels have a strong correlation with adjacent pixels in each picture, pictures are compressed using the correlation in each picture. Furthermore, the consecutive pictures are compressed using a correlation between pixels in different pictures because the consecutive pictures have the strong correlation between pixels.
Here, compression using a correlation between pixels in different pictures and a correlation between pixels in a picture is referred to as inter coding, whereas compression using only the correlation between pixels in a picture without using the correlation between pixels in different pictures is referred to as intra coding. The inter coding that uses the correlation between pictures can achieve a compression rate higher than that of the intra coding.
Furthermore, in accordance with MPEG-1, MPEG-2, MPEG-4, and MPEG-4 AVC (H.264), each image includes blocks (or a macroblock as a generic concept of the blocks) that are groups of pixels in a two-dimensional rectangular area, and the inter coding and the intra coding are switched per block.
On the other hand, with widespread high-speed network environment using Asymsatisfiedric Digital Subscriber Lines (ADSLs) and optical fibers, general households can transmit and receive information at a bit rate over several Mbit/s. Furthermore, it is expected that information can be transmitted and received at several tens of Mbit/s in the next few years.
Thereby, the expectation is that with the image coding technique, not only companies using dedicated lines but also general households will introduce video telephony services and teleconferencing systems that guarantee the television broadcast quality and HDTV broadcast quality.
The high-speed network using an ADSL or an optical fiber is not a bandwidth-guaranteed dedicated network that is expensive and is targeted at companies but a best-effort network that is inexpensive and is to be used in common by users. What is determined in the best-effort network is the upper limit of a sum of bit rates used by the users at the time. Thus, the bit rate available per user decreases at the time when the number of users increases, whereas the bit rate available per user increases at the time when the number of users decreases. In other words, there is a feature that the available bit rate largely changes depending on a time.
Furthermore, moving images includes images that can be very easily compressed, such as a whole-colored solid image (image having the same color and the same brightness on an entire screen). In contrast, there are images that are very difficult to be compressed due to no correlation between pixels, such as white noise. Thus, it is important to stably code moving images that considerably differ in ease of compression, without any substantial image degradation.
Before coding a moving image that is difficult to be compressed, the conventional image coding apparatus converts a resolution thereof to a lower resolution and reduces the number of pixels to be coded (PTL 1).
FIG. 24 is an explanation drawing for describing a conventional moving image coding apparatus. The left column of FIG. 24 illustrates image sizes (resolutions) of an image to be coded, that is, resolutions at which the image is actually coded. On the other hand, the right column of FIG. 24 illustrates respective sizes of images displayed by a display apparatus, where the images are obtained by decoding coded images and enlarging respective sizes of the decoded images.
Here, the images to be coded include an image that is relatively easy to be compressed and an image that is relatively difficult to be compressed. The moving image that is relatively easy to be compressed (top of FIG. 24) is coded without changing the resolution, the coded image is decoded by an image decoding apparatus, and the decoded image is displayed in the original resolution.
In contrast, the moving image that is relatively difficult to be compressed is coded with less number of pixels (lower resolution) after vertically and horizontally reducing the number of pixels (resolution) to ¾ (middle of FIG. 24) or to ½ (bottom of FIG. 24). Furthermore, the display image decoded by the image decoding apparatus is displayed with the number of pixels (resolution) same as that of the original moving image by increasing the resolution by 4/3 times or 2 times.
Even when an image that is difficult to be compressed is coded, coding the less number of pixels (resolution) can prevent the substantial image degradation. However, even when an image having the less number of pixels is enlarged into an image having the larger number of pixels, since it is not possible to achieve the representation over the fineness (minuteness) that can be represented using the less number of pixels that have been coded, the obtained image becomes more blurred than the image that has been coded at the same magnification.
Furthermore, when a moving image that is difficult to be compressed is coded, there is a method of not only changing the resolution of the image, but also controlling a frame rate of the image that is the number of frames per unit time of an image to be coded, and the quantization step when coding per block.
FIG. 25 is a block diagram illustrating a conventional image coding apparatus in which a resolution, a frame rate, and a quantization step are dynamically changed. As illustrated in FIG. 25, the conventional image coding apparatus mainly includes a resolution changing circuit 501, a frame rate changing circuit 502, a moving image coding circuit 503, a quantization step control circuit 506, a frame rate control circuit 507, and a resolution setting circuit 508.
The resolution setting circuit 508 determines the resolution at which an image is coded according to the difficulty in compression as described in PTL 1, and notifies the resolution changing circuit 501 of a resolution signal S64 indicating the resolution of the image to be coded. The resolution changing circuit 501 converts a video signal S60 that is an input from a video input terminal 500 at a predetermined resolution into a video signal S61 having the resolution notified from the resolution setting circuit 508.
The frame rate control circuit 507 notifies the frame rate changing circuit 502 of a frame rate signal S67 indicating the frame rate of the image to be coded. The frame rate changing circuit 502 changes the frame rate of the video signal S61 in which the resolution has been changed into the frame rate notified from the frame rate control circuit 507.
The quantization step control circuit 506 notifies the moving image coding circuit 503 of a quantization step signal S66 indicating the quantization step for quantization by the moving image coding circuit 503. The moving image coding circuit 503 quantizes and codes a video signal S62 in which the frame rate has been changed using the quantization step notified from the quantization step control circuit 506, and outputs a bitstream S63 to a bitstream output terminal 504.
The quantization step control circuit 506 determines the quantization step, based on a target coding bit rate, the code amount of the bitstream S63 outputted from the moving image coding circuit 503, and a value indicated by the frame rate signal S67. Furthermore, the frame rate control circuit 507 determines a frame rate according to a value indicated by the quantization step signal S66 determined by the quantization step control circuit 506.
With such a configuration, the resolution setting circuit 508 determines a resolution according to the difficulty in compressing an image. Furthermore, when the degree of difficulty in compressing an image in which the resolution has been changed is changed, the conventional image coding apparatus dynamically controls the frame rate and the quantization step, and codes the image at a target bit rate.